FLIR-to-missile boresight correlation and non-uniformity compensation of the missile seeker

ABSTRACT

The present invention provides for simple and streamlined boresight correlation of FLIR-to-missile video. Boresight correlation is performed with un-NUCed missile video, which allows boresight correlation and NUC to be performed simultaneously thereby reducing the time required to acquire a target and fire the missile. The current approach uses the motion of the missile seeker for NUCing to produce spatial gradient filtering in the missile image by differencing images as the seeker moves. This compensates DC non-uniformities in the image. A FLIR image is processed with a matching displace and subtract spatial filter constructed based on the tracked scene motion. The FLIR image is resampled to match the missile image resolution, and the two images are preprocessed and correlated using conventional methods. Improved NUC is provided by cross-referencing multiple measurements of each area of the scene as viewed by different pixels in the imager. This approach is based on the simple yet novel premise that every pixel in the array that looks at the same thing should see the same thing. As a result, the NUC terms adapt to non-uniformities in the imager and not the scene.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the boresight correlation and non-uniformitycompensation that must occur before target handover from the FLIR to themissile seeker, and more specifically to a method of correlatingun-NUCed missile video to the FLIR and an improved scene based method ofnon-uniformity compensation.

2. Description of the Related Art

In guided missile systems, certain calibrations must be performed beforetarget handover from the target designation device such as a forwardlooking infrared (FLIR) to the missile seeker. These include performinga boresight correlation to correct for any imprecise mechanic alignmentsbetween the FLIR and missile seeker and performing a non-uniformitycompensation to reduce pixel-to-pixel response non-uniformities in theseeker's IR imager. These adjustments improve the missile's chance ofacquiring and then tracking the target to impact.

As shown in FIGS. 1 and 2, a soldier 10 uses a shoulder launched missilesystem 12 to target and launch a missile 14 to destroy a tank 16. Themissile system is provided with a forward looking infrared (FLIR) system18 that includes a high resolution imager that acquires a highresolution image 20 of the scene within its FOV 22. The missile 14 isoutfitted with a missile seeker 24 having a low resolution imager thatacquires a low resolution image 26 of the scene within its FOV 28. TheFLIR and missile seeker communicate information over a channel such as adata bus 30. As illustrated, the FLIR and seeker's FOVs are not alignedand the non-uniformity of the IR imager is such as to almost completelyobscure the tank 16 within the low resolution image 26.

Automated systems of performing boresight correlation have been in thefield for decades, e.g. LANTIRN. Substantial knowledge has been acquiredconcerning robust approaches, for example, edge pre-filtering and maskedbi-level correlation for textured scenes, with spot-feature basedcorrelation for bland scenes. However, these methods require arelatively uniform response across the missile seeker image. In aground-to-ground missile, image uniformity is achieved by completelycooling down and non-uniformity compensating the seeker's IR imagerbefore boresight correlation begins. Once all three serial operationsare completed and the data handed over to the missile seeker, thesoldier will see seeker video with minimal noise and centered on hisFLIR video. This requires time, which exposes the soldier to enemy fireand endangers the mission of destroying the target.

Although non-uniformity compensation (“NUCing”) is required to provideimages of sufficient quality to perform boresight correlation, itsprimary function is to reduce the fixed-pattern noise level of missileIR video for acquisition by the soldier and possibly during trackingpost-launch. NUC is required because the response of the IR imager on apixel by pixel basis can change dramatically over time and based onenvironmental and operating conditions. The NUC procedure should beeffective at removing or at least substantially reducing thepixel-to-pixel variability in the response of the imager, should be donequickly and should require minimal operator involvement (“knobtweaking”).

The predominant approach for NUCing uses a blurred version of the scenecreated optically, through motion of the IR seeker, or through temporalaveraging. The premise is that on-average all pixels should all pixelsshould see the same value. Based on this assumption, any high frequencycomponents that are detected in the blurred image for each pixel aredeemed to be the result of non-uniform pixel response. The blurred imageis corrected to remove the high frequency components. The samecorrection is then applied to the subsequent non-blurred image. Thisapproach is serviceable for relatively “flat” imagery but struggles withscenes which contain significant content at high spatial frequencies.These may be perceived as non-uniformities and “compensated” producingscene and body-motion dependent artifacts.

Once NUCing is complete and missile video with a low enough noise levelis available, boresight correlation can be performed to correct anymisalignment between the missile and FLIR. Boresight correlation istypically performed by first resampling the FLIR image to a coarserresolution to match the seeker's resolution. Next, the NUCed missileimage is sent across the data bus and correlated against the re-sampledFLIR image. The correlation process is well known to those skilled inthe art. The position of the correlation peak indicates the position ofthe center of the missile image relative to the FLIR's boresight (centerof the FLIR's field of view). Once this relative boresight offset isknown, the target position and size from the FLIR can then be mappedinto the missile image.

To improve the effectiveness of these weapons systems and to protect thesoldiers, techniques for simplifying and streamlining the boresightcorrelation are needed. Furthermore, techniques for NUC that providehigher bandwidth, improved noise reduction and minimum user involvementthat is applicable to severely non-uniform scenes would further improvethe efficacy of the weapons system and other IR imaging systems.

SUMMARY OF THE INVENTION

The present invention provides for simple and streamlined boresightcorrelation of FLIR-to-missile video and for higher bandwidth, higherSNR, user friendly NUC of IR imagers. The NUC process also provides thecapability of displaying enhanced imagery.

This is accomplished in part with a method that performs boresightcorrelation with un-NUCed missile video. This allows boresightcorrelation and NUC to be performed simultaneously thereby reducing thetime required to acquire a target and fire the missile. The currentapproach uses the motion of the missile seeker for NUCing to producespatial gradient filtering in the missile image by differencing imagesas the seeker moves. This compensates DC non-uniformities in the image.Although there are residual artifacts from scene content and gainnon-uniformities, they are not sufficiently severe to interfere withcorrelation when proper pre-filtering is used. A FLIR image is processedwith a matching displace and subtract spatial filter constructed basedon the tracked scene motion. The FLIR image is resampled to match themissile image resolution, and the two images are preprocessed andcorrelated using conventional methods.

Although this method of boresight correlation can be used in tandem withconventional techniques for NUC, additional advantages of increasedbandwidth, improved SNR of the missile video and less operatorinvolvement can be realized with improved scene-based NUC. The currentapproach works by cross-referencing multiple measurements of each areaof the scene as viewed by different pixels in the imager. This approachis based on the simple yet novel premise that every pixel in the arraythat looks at the same thing should see the same thing. As a result, theNUC terms adapt to non-uniformities in the imager and not the scene.

In scene-based NUC, a sequence of temporal images from the missile videoare captured and compensated with apriori NUC terms assuming they exist.The FOV motion across the scene is tracked and the compensated imagesare registered and summed together to form a “registered sum image”. A“registered count image” records the number of pixels (or the totalpixel weights) contributing to each pixel in the registered sum image. A“registered average image” is created by dividing the “registered sumimage” by the “registered count image”. This image constitutes the bestavailable estimate of the actual scene content. The image is, to someextent, contaminated by uncompensated non-uniformities but these will becancelled. The “registered average image” is backward mapped by cuttingout frames from the collage that correspond to each of the input imagesto produce estimates of the ideal input image, e.g. assuming perfectNUC. Each of the estimates is subtracted from the respective compensatedimage to create a sequence of “unexplained difference” images. An Nthorder correlation of these differences against the input images producesa pixel-by-pixel estimate of NUC terms needed to correct any remaininguniformities. These NUC terms are combined with any a priori NUC terms,e.g. factory calibration or terms resulting from prior iterations, andused to compensate the missile video. In the case of simple DC offsetcompensation, this combination is a simple sum. Typically, the processwill iterate a number of times to beat down the noise level. The numberof iterations can be fixed, based on convergence criteria of the NUCterms or on time allowed the imaging system.

This process can be enhanced by keeping track of the variability in theforward mapping (creating the registered average image) and/or backwardmapping (estimating the ideal input images) and weighting those imagerand or scene pixels with less variability more heavily. This has theeffect of favoring those imager pixels (camera pixels) with the mostconsistent response for estimating scene content during the “forwardmapping”, and of favoring those areas of the scene (scene pixels) whichare most spatially and temporally consistent in estimating pixelresponses during “backward mapping”. Spatial consistency enters in tothis variability through aliasing in the backward mapping: spatiallyconsistent areas produce less aliasing, hence less variability in thebackward mapping.

The process can also be enhanced by storing the registered average imageat greater than the seeker pixel resolution. Specifically, use of simplenearest neighbor mapping (vs. interpolation) to expand the number ofpixels in the FOV has proven quite beneficial. Phase variations in theinput images cause aliased frequency components from the replication tocancel while aliasing artifacts that are consistent with the actualbeyond-Nyquist frequencies in the input image add constructively and arepreserved. Anti-aliased interpolation methods for creating the enlargedscene tend to diminish this desirable effect.

The process can be computationally simplified if the variability is nottracked and the non-uniformity of the imager is assumed to be zeroorder, e.g. just the offset. In this case, the individual frames neednot be retained to support the computational iteration, only the stackedaverage, the registered sum, and the registered count images (the latterbeing fixed throughout the iterations) are required. Ideal stackedaverages are computed by convolving the FOV motion trace (f) against theregistered average image, residual non-uniformities are identified bydifferencing this with the input stacked average, and residualdifferences (DC non-uniformity estimates) are propagated into theregistered average image by correlating them against the FOV motiontrace (f) and adding the incremental result into the registered sum.

These and other features and advantages of the invention will beapparent to those skilled in the art from the following detaileddescription of preferred embodiments, taken together with theaccompanying drawings, in which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1, as described above, is a block diagram of a FLIR and missilesystem;

FIG. 2, as described above, illustrates a frame of FLIR video and aframe of un-NUCed missile video prior to boresight correlation;

FIG. 3 is a block diagram of a parallel process of non-uniformitycompensation and boresight correlation for target handover in accordancewith the present invention;

FIG. 4 is a block diagram of boresight correlation using un-NUCedmissile video and auto correlation of the un-NUCed missile video;

FIG. 5 is a diagram of an iterated NUC process;

FIG. 6 is a flowchart for estimating the NUC error terms;

FIGS. 7 a through 7 f are illustrations of estimation process;

FIG. 8 is a flowchart of an alternate embodiment that tracks variabilityduring the forward and backward mapping processes;

FIG. 9 is a diagram of an alternate embodiment in which the registeredaverage image is stored at greater than seeker pixel resolution;

FIG. 10 is a flowchart of a computationally efficient embodiment for azero order system; and

FIG. 11 is a flowchart for using the information provided by NUCing todisplay an enhanced image.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides for simple and streamlined boresightcorrelation of FLIR-to-missile video and for higher bandwidth, higherSNR, user friendly NUC of IR imagers.

As shown in FIG. 3, this is accomplished in part with a method thatperforms boresight correlation with un-NUCed missile video 40. Thisallows boresight correlation 42 with the FLIR video 44 and NUC 46 on themissile video 40 to be performed simultaneously thereby reducing thetime required to perform target handover 48 from the FLIR to the missileseeker and fire the missile. NUC is not required for target handover,however it is useful to improve the SNR of the missile video. The NUCprocess also provides the capability of displaying enhanced, e.g., noisefiltered and super resolved, imagery on an operator display 50.

The current approach to boresight correlation temporally processes themissile video from the IR imager to locate edges in an IR image,spatially processes the FLIR video in accordance with an estimate of thescene motion to locate at least some of the same edges in a FLIR image,and correlates the missile and FLIR images. More specifically, seekermotion converts spatial edges in an image into temporal signatures in asequence of images. The signatures are used to detect the edges using,for example, frame differencing. The FLIR video is processed spatiallyin accordance with scene motion using, for example, a displace andsubtract filter, to locate at least some of the same edges in a FLIRimage. The missile and FLIR images (edge maps) are correlated to provideboresight correlation.

As shown in FIG. 4, in an exemplary embodiment successive images (framesof missile video 40) are delayed 52 and subtracted 54 from the nextimage to form an edge map 56. This cancels DC non-uniformities in theimage. Although there are residual artifacts from scene content and gainnon-uniformities, they are not sufficiently severe to interfere withcorrelation when proper pre-filtering is used. Higher order differencescan be formed to improve the edge map at the cost of additionalcomputing complexity. A FLIR image (frame of FLIR video 44) is processedwith a matching displace and subtract spatial filter 58 constructedbased on a scene motion estimate 59 from, for example, gimbal positionof the missile seeker and integrated inertial rate sensors and/orautocorrelation of the missile video. The FLIR image is resampled 60 tomatch the missile image resolution, and the two images are preprocessedand correlated 61 using conventional methods. The missile images (edgemaps) are delayed 62 and autocorrelated 63 with the next missile imageto provide registration for NUC. By extracting the registration from theedge map rather than waiting for the boresight correlation to determineregistration, NUC can be performed simultaneously with boresightcorrelation thereby reducing the time required to perform targethandover.

Although this method of boresight correlation can be used in tandem withconventional techniques for NUC, additional advantages of increasedbandwidth, improved SNR of the missile video and less operatorinvolvement can be realized with improved scene-based NUC. The currentapproach as illustrated in FIGS. 5-8 works by cross-referencing multiplemeasurements of each area of the scene as viewed by different pixels inthe imager. This approach is based on the simple yet novel premise thatevery pixel in the array that looks at the same thing should see thesame thing. As a result, the NUC terms adapt to non-uniformities in theimager and not the scene.

As shown in FIG. 5, the basic flow diagram for NUC is to capture rawinput images of the missile video (step 64), compensate each frame on apixel-by-pixel basis using the most recent NUC terms (step 65) andestimate NUC error terms (step 66) based on the compensated images. TheNUC error terms are combined with the current NUC terms (step 67) toupdate the NUC terms (step 68) that are used for the next iteration. TheNUC terms may be initialized with a priori NUC terms (factorycalibration) (step 69). Typically, the process will iterate a number oftimes on the compensated images to beat down the noise level. The numberof iterations can be fixed, based on convergence criteria of the NUCterms or on time allowed by the imaging system.

As shown in FIGS. 6 and 7, the NUC error terms are estimated byproviding the sequence of compensated images 70 from the missile video(step 71) and tracking the associated FOV motion (f) 72 across the scene(step 73). The images are registered using the FOV motion and summedtogether to form a “registered sum image” (step 74). A “registered countimage” 75 records the number of pixels (or, as described below, thetotal pixel weights) contributing to each pixel in the registered sumimage (step 76). A “registered average image” 77 is created by dividingthe “registered sum image” by the “registered count image” (step 78).This image constitutes the best available estimate of the actual scenecontent. The image is, to some extent, contaminated by uncompensatednon-uniformities but these will be cancelled. Together steps 73, 74,76and 78 constitute the “forward mapping”. Using the scene motion acrossthe FOV (f¹) 80, the registered average image 77 is “backward mapped”(step 82) by cutting out frames 83 a, 83 b, 83 c, and 83 d from thecollage that correspond to each of the input images to produce estimates84 a, 84 b, 84 c, and 84 d of the ideal input image, e.g. assumingperfect NUC. Each of the estimates is subtracted from the respectivepreviously compensated image 70 to create a sequence of “unexplaineddifference” images 85 (step 86). An Nth order correlation (step 88) ofthese difference images 85 against the respective estimated imagesproduces a pixel-by-pixel estimate of NUC error terms 90 needed tocorrect any remaining uniformities. In the case of a 0^(th) ordercorrelation, (assuming only DC offset, no gain terms), the differenceimages 85 are simply averaged pixel-to-pixel without reference to theinput images. In this particular embodiment, incremental errors in theNUC are accumulated at each iteration. Particularly for 0^(th) order (DCcompensation) it may be preferable to reference back to the originalimages instead of the iteratively compensated images so that thecomplete NUC term is calculated at each iteration instead of a NUC errorterm.

This process can be enhanced by keeping track of the variability in theforward mapping (creating the registered average image) and/or backwardmapping (estimating the ideal input images) and weighting those imagerpixels and/or scene pixels with less variability more heavily. This hasthe effect of favoring those imager pixels with the most consistentresponse for estimating scene content during the “forward mapping”, andof favoring those areas of the scene (scene pixels) which are mostspatially and temporally consistent in estimating pixel responses during“backward mapping”. Spatial consistency enters in to this variabilitythrough aliasing in the backward mapping: spatially consistent areasproduce less aliasing, hence less variability in the backward mapping.

With reference to FIGS. 6 and 8, backward mapping variability is trackedby making an imager pixel stationarity map 102 available (step 104)during the creation of the registered sum image. The map values are ameasure of the variability of the response of each pixel in the imager.Pixels that respond consistently will be assigned a small variabilityand pixels such as “blinkers” that respond inconsistently will beassigned a larger variability. Weights W_(i) for each imager pixel arethen derived from the stationarity map 102 (step 106). For example, ifthe measure of variability is the variance of the pixel response, theweight is suitably inversely proportional to the standard deviations. Itfollows that pixels that exhibit a higher degree of stationarity, e.g. alower variance, are assigned a larger weight and vice-verse. The weightsmay be biased by adding a constant value to the standard deviation. Thisis done to keep the weight above the noise level when only a few samplescontribute to the pixel. Furthermore, the constant is set to a largeenough value to assure convergence to a set of NUC terms. The map, hencethe weights are typically but not necessarily initialized to a uniformset of values to start the iterative process. The iterative process willidentify the good and bad pixels and the stationary and non-stationaryportions of the video, and will rebalance the weights accordingly.

The “registered weighted sum image” is now formed as the weighted sum ofthe registered image values I_(i), i.e. (ΣW_(i)I_(i)) for each pixel(step 108). As such, the contribution from “good” pixels is emphasizedand the contribution from “bad” pixels is de-emphasized. The weightsW_(i) are also used to form a “registered weighted count image” (step110) in which each entry corresponding to a scene pixel is the sum ofthe weights of the contributing imager pixels (ΣW_(i)). A scene pixelstationarity map 112 provides a measure of the temporal and spatialstationarity of each scene pixel (step 114). To a lesser but stillsignificant extent, this map also incorporates the effects of bad imagerpixels such as “blinkers”. This map can be generated by, for example,computing the weighted variance at each scene pixel in the registeredsum image, i.e. the weighted variance of the registered imager pixelsthat contribute to each scene pixel. Assuming an ideal imager pixelresponse, the scene map accounts for both temporal variations (video)and spatial variations (seeker motion) in the images. Alternately,spatial and/or temporal high pass filtering can be performed on theinput images prior to forming the registered sum image to provide local“flatness” estimates. The spatial filters are then constructed withknowledge of the FOV motion.

The registered average image is created by dividing the “registeredweighted sum image” by the “registered weighted count image” (step 116).By emphasizing the “good” pixels and de-emphasizing the “bad” pixels,the estimated input images cut out of the collage are better estimatesof the ideal input images. The registered average image is backwardmapped and the unexplained difference image formed as before (step 118).An Nth order correlation (step 120) of these difference images againstthe respective estimated images weighted by the scene pixel map 112produces a pixel-by-pixel estimate of NUC error terms 90 needed tocorrect any remaining non-uniformities. The imager pixel stationaritymap 102 is updated to be the variance of the residual at each pixelafter the correlation is performed (step 121). For a zero order system,this defaults to the variance of the unexplained differences at eachpixel

In another embodiment, as shown in FIG. 9 the NUC estimation process canalso be enhanced by storing the registered average image at greater thanthe seeker pixel resolution. Specifically, each compensated image 70 isupsampled by a factor of 2, 4, 8, etc. and values are assigned to thepixels 130. As a result, the registered sum image, hence registeredaverage image are registered on a pixel level of the upsampledcompensated images, which corresponds to sub-pixel resolution of theoriginal images. The use of simple nearest neighbor mapping (vs.interpolation) to assign values to the expanded pixels in the FOV hasproven quite beneficial. Phase variations in the compensated imagescause aliased frequency components from the replication to cancel whilealiasing artifacts that are consistent with the actual beyond-Nyquistfrequencies in the input image add constructively and are preserved.Anti-aliased interpolation methods for creating the enlarged scene tendto diminish this desirable effect. Estimating the registered averagescene at a higher resolution typically has the effect of improvingperformance in areas of high spatial frequency content by diminishingaliasing in the backward mapping. It allows sub-pixel registration ofthe backward mapped images. The corresponding regions (2×2 at 2×resolution) are averaged to produce each backward mapped pixel.

The iterative process can be computationally simplified if thevariability is not tracked in either the forward or backward mappingprocesses and the non-uniformity of the imager is assumed to be zeroorder, e.g. just the DC offset. In this case, the individual images neednot be retained to support the computational iteration, only the stackedaverage of the raw or apriori compensated images, the registered sum,and the registered count images (the latter being fixed throughout theiterations) are required. Ideal stacked averages are computed byconvolving the FOV motion trace (f) against the registered average image(or equivalent correlating against the scene motion across the FOV f¹),residual non-uniformities are identified by differencing this with theinput stacked average, and residual differences (DC non-uniformityestimates) are propagated into the registered average image bycorrelating them against the FOV motion trace (f) and adding theincremental result into the registered sum. Since convolution isequivalent to correlating against a reversed mask, one operationcollects the portions of the scene seen by each pixel, and onedistributes the residual errors from the image to the scene. Since thereference point reverses in these two case (fixed in the imager FOV vs.fixed in the scene), the correlation mask reverses.

As shown in FIG. 10, NUCing is accomplished by capturing a temporalsequence of input images (step 150), compensating each image usingapriori NUC terms 151 (if they exist) such as factory calibration terms(step 152), and tracking the associated FOV motion (f) across the scene(step 154). The images are registered using the FOV motion and summedtogether to form an initial “registered sum image” (step 156). Thenumber of pixels contributing to each pixel in the registered sum imageis recorded in a “registered count image” (step 158). The “registeredsum image” is divided by the “registered count image” to form the“registered average image” (step 160). This image constitutes the bestavailable estimate of the actual scene content. The image is, to someextent, contaminated by uncompensated non-uniformities but these will becancelled.

Up to this point, the NUC process is the same as that originallydiscussed with respect to FIGS. 5 and 6. By assuming a zero ordercompensation and forgoing tracking variability, the computationalprocess may be simplified. Instead of having to iterate the entireprocess, all that is required is an iteration around updating theregistered sum image. First, the FOV motion trace f162 is convolvedagainst the registered average image to form an ideal stacked average(step 164). The ideal stacked average is differenced with a stackedaverage (average of the compensated images (step 166)) to form acomposite unexplained difference image (DC non-uniformity estimates)(step 168). The composite image is output as the NUC error terms 170,which are combined with the updated NUC terms 172 from the previousiteration to form current updated NUC terms 172 (step 173).

Now, instead of having to compensate the input images with the updatedNUC terms and create the registered sum image anew, the NUC error termsare simply correlated against the FOV motion trace f to form aregistered sum error (step 174) and added to the registered sum image(step 176) to update the registered sum image, which in turn updates theregistered average image. This inner loop repeats for a certain numberof iterations, time or until the updated NUC terms 172 converge.

As shown in FIG. 11, the NUC process also provides the capability ofproviding enhanced imagery to an operator display 200 or an automatedimage process 202. For each compensated image, the scene pixelstationarity map 112 is used to select the number of images N_(i) tomerge at each scene pixel (step 204). The more stable the scene pixelthe larger N_(i). Then for each input image frame, the FOV motion 72 isused to select the most recent N_(i) contributors at each imager pixel(step 206). These two steps essentially determine the width of atemporal averaging filter for each pixel and the motion compensation foreach pixel. The filter can be a straight average, weighted average orsome higher order filter. Armed with this information, a compositeaverage image is created for each frame from the compensated images(step 208). The composite image is enhanced in two distinct ways. First,in areas of the image that are relatively stable, significant noisereduction is realized by merging a large number of images. Furthermore,in areas of the image that are varying significantly, detail ispreserved by merging only a few or no images. Second, particularly inthe enhanced resolution method of FIG. 9, the merging of informationfrom adjacent frames can reconstruct scene content in the compositeimage that was not originally there in that is exceeded the Nyquistlimit for an individual frame. Just as a scanning linear array is oftenused to provide image resolution in the cross axis, moving a twodimensional imager can provide enhanced resolution in both axes.

While several illustrative embodiments of the invention have been shownand described, numerous variations and alternate embodiments will occurto those skilled in the art. Such variations and alternate embodimentsare contemplated, and can be made without departing from the spirit andscope of the invention as defined in the appended claims.

1. A method of providing target handover from a FLIR to a missiletracker, comprising: receiving missile video from the tracker's IRimager; receiving FLIR video from the FLIR; performing a boresightcorrelation of the un-NUCed missile video to the FLIR video to align theIR imager to the FLIR; and performing non-uniformity compensation (NUC)on the missile video.
 2. The method of claim 1, wherein the IR image ismounted on a seeker, boresight correlation being performed by, using thetemporal motion of the seeker over a plurality of frames of missilevideo to form a first edge map; measuring scene motion in the missilevideo; spatially processing images in the FLIR video in accordance withscene motion to locate at least some of the same edges to form a secondedge map; and correlating the first and second edge maps.
 3. The methodof claim 2, wherein the first edge map is formed by differencing imagesin the missile video.
 4. The method of claim 3, wherein a first frame ofthe missile video is subtracted from a second frame to form the firstdifference image.
 5. The method of claim 2, wherein the FLIR video isspatially processed with a displace and subtract filter in accordancewith the measured scene motion.
 6. The method of claim 1, wherein thenon-uniformity compensation is performed by, registering a temporalsequence of input images from the missile video having different FOVs tocreate an estimate of an input scene; backward mapping the estimatedinput scene to the different FOVs to create an estimate input image foreach; comparing each said estimate input image to the associated inputimage to estimate pixel-by-pixel response non-uniformity compensation(NUC) terms for the IR imager; applying the NUC terms to the images togenerate compensated images.
 7. The method of claim 6, wherein theestimate of the input scene is created by, adding the registered inputimages together to create a registered sum image; accumulating thenumber of input images that contribute to each pixel in the scene in aregistered count image; dividing the registered sum image by theregistered count image to form a registered average image.
 8. The methodof claim 8, further comprising keeping track of a variability of theresponse of the imager pixels in the forward mapping to update theregistered sum image, wherein imager pixels that provide the mostconsistent response contribute most heavily to the registered sum andregistered count images, hence the estimate of the input scene
 9. Themethod of claim 7, wherein the registered average image is maintained ata higher resolution than the input image.
 10. The method of claim 9,wherein a nearest neighbor mapping is used to expand the number ofpixels in the FOV of the input image into the registered average image,phase variation in the input images causing aliased frequency componentsfrom the mapping to cancel while artifacts that are consistent withbeyond-Nyquist frequencies in the input image add constructively and arepreserved.
 11. The method of claim 6, wherein the estimated input sceneis backward mapped to each input image by cutting out a frame in theestimated input scene that corresponds to that input image's field ofview.
 12. The method of claim 11, further comprising keeping track of avariability of the scene pixels in the backward mapping to update theestimated NUC terms, wherein those areas of the estimate of the inputscene which are temporally stable and locally flat contribute mostheavily to the estimate of the NUC term.
 13. The method of claim 6,wherein the NUC terms are estimated by, differencing respective pairs ofinput images and estimated input images to form a sequence ofunexplained difference images; performing an Nth order correlation onsaid unexplained difference images to form NUC error terms; andcombining the NUC error terms with the NUC terms to update the NUCterms.
 14. The method of claim 7, wherein said backward mappingcomprises convolving the FOV motion trace against the registered averageimage to form an ideal stacked average that is compared to a stackedaverage of the input images to form a composite unexplained differenceimage that is output as the NUC error terms to update the NUC terms,said NUC error terms being correlated again the FOV motion trace andadded to the registered sum image to update the registered average imagefor another iteration on the NUC error terms.
 15. The method of claim 6,wherein the steps are repeated on the compensated images to improve theestimate of the NUC terms.
 16. The method of claim 6, furthercomprising: computing a scene pixel variability map; for each pixel(i,j) in the scene, determine the number N(i,j) of images to merge basedon the variability map; for each pixel in the scene, selecting the mostrecent N(i,j) images that contribute to said pixel, and merging theN(i,j) values from the selected images for each pixel to form acomposite image.
 17. The method of claim 16, wherein a video sequence ofsaid composite images is displayed.
 18. A method of correlatinguncompensated IR video to reference video, comprising: using thetemporal motion of the IR imager over a plurality of frames of IR videoto form a first edge map; measuring scene motion in the IR video;spatially processing images in the reference video in accordance withthe scene motion to locate at least some of the same edges to form asecond edge map; and correlating the first and second edge maps.
 19. Themethod of claim 18, wherein the first edge map is formed by differencingimages in the missile video.
 20. The method of claim 19, wherein a firstframe of the IR video is subtracted from a second frame to form thefirst difference image.
 21. The method of claim 18, wherein thereference video is spatially processed with a displace and subtractfilter in accordance with the measured scene motion.
 22. A method ofenhancing video images, comprising: performing non-uniformitycompensation (NUC) on a sequence of input images; providing a scenepixel stationarity map for each input image from the NUC; providing ascene motion estimate for each input image from the NUC; for eachcompensated image, using the scene pixel stationarity map to select anumber N_(i) of images to merge at each scene pixel; using the scenemotion estimate to select the most recent N(i,j) contributors at eachimage pixel (i,j); and creating a composite image by merging the N(i,j)contributors from the compensated images at each image pixel.
 23. Themethod of claim 22, wherein a video sequence of said composite images isdisplayed.
 24. A method of non-uniformity compensation (NUC) for apixilated IR imager, comprising: registering a temporal sequence ofinput images from the IR imager having different FOVs to create anestimate of an input scene; backward mapping the estimated input sceneto the different FOVs to create an estimate input image for each;comparing each said estimate input image to the associated input imageto estimate pixel-by-pixel response non-uniformity compensation (NUC)terms for the IR imager; and applying the NUC terms to the images togenerate compensated images.
 25. The method of claim 24, wherein theestimate of the input scene is created by, adding the registered inputimages together to create a registered sum image; accumulating thenumber of input images that contribute to each pixel in the scene in aregistered count image; dividing the registered sum image by theregistered count image to form a registered average image.
 26. Themethod of claim 25, further comprising keeping track of a variability ofthe response of the imager pixels in the forward mapping to update theregistered sum image, wherein imager pixels that provide the mostconsistent response contribute most heavily to the registered sum andregistered count images, hence the estimate of the input scene
 27. Themethod of claim 25, wherein the registered average image is maintainedat a higher resolution than the input image.
 28. The method of claim 27,wherein a nearest neighbor mapping is used to expand the number ofpixels in the FOV of the input image into the registered average image,phase variation in the input images causing aliased frequency componentsfrom the mapping to cancel while artifacts that are consistent withbeyond-Nyquist frequencies in the input image add constructively and arepreserved.
 29. The method of claim 24, wherein the estimated input sceneis backward mapped to each input image by cutting out a frame in theestimated input scene that corresponds to that input image's field ofview.
 30. The method of claim 29, further comprising keeping track of avariability of the scene pixels in the backward mapping to update theestimated NUC terms, wherein those areas of the estimate of the inputscene which are temporally stable and locally flat contribute mostheavily to the estimate of the NUC term.
 31. The method of claim 24,wherein the NUC terms are estimated by, differencing respective pairs ofinput images and estimated input images to form a sequence ofunexplained difference images; performing an Nth order correlation onsaid unexplained difference images to form NUC error terms; andcombining the NUC error terms with the NUC terms to update the NUCterms.
 32. The method of claim 25, wherein said backward mappingcomprises convolving a FOV motion trace against the registered averageimage to form an ideal stacked average that is compared to a stackedaverage of the input images to form a composite unexplained differenceimage that is output as the NUC error terms to update the NUC terms,said NUC error terms being correlated again the FOV motion trace andadded to the registered sum image to update the registered average imagefor another iteration on the NUC error terms.
 33. The method of claim24, wherein the steps are repeated on the compensated images to improvethe estimate of the NUC terms.
 34. A method of non-uniformitycompensation (NUC) for a pixilated IR imager, comprising: capturing atemporal sequence of input images from the IR imager having differentFOVs; compensating each input image with apriori NUC terms if they existto form compensated images; tracking a FOV motion across the scene;registering and adding the input images together to create a registeredsum image; accumulating the number of input images that contribute toeach pixel in the scene in a registered count image; dividing theregistered sum image by the registered count image to form a registeredaverage image; using the FOV motion to backward map the registeredaverage image to the different FOVs to create an estimated input imagefor each; differencing respective pairs of compensated images andestimated input images to form a sequence of unexplained differenceimages; performing an Nth order correlation on said unexplaineddifference images to form NUC error terms; and combining the NUC errorterms with the NUC terms from the previous iteration to update the NUCterms.
 35. The method of claim 34, further comprising keeping track of avariability of the response of the imager pixels to update theregistered sum and registered count images and a variability of scenepixels to weight the Nth order correlation.
 36. A method ofnon-uniformity compensation (NUC) for a pixilated IR imager, comprising:capturing a temporal sequence of input images from the IR imager havingdifferent FOVs; compensating each input image with apriori NUC terms toform compensated images; tracking a FOV motion across the scene;registering and adding the input images together to create a registeredsum image; accumulating the number of input images that contribute toeach pixel in the scene in a registered count image; dividing theregistered sum image by the registered count image to form a registeredaverage image; convolving the FOV motion against the registered averageimage to form an ideal stacked average; comparing the ideal stackedaverage to a stacked average of the compensated images to form acomposite unexplained difference image that is output as the NUC errorterms to update the NUC terms; correlating said NUC error terms againstthe FOV motion trace to form a registered sum error; and adding theregistered sum error to the registered sum image to update theregistered average image for another iteration on the NUC error terms.